This invention relates to network switch systems, and more particularly to phase adjustment without pointer manipulation for Synchronous Optical NETwork (SONET) frames.
Very high bandwidth network links such as fiber-optic backbone trunks are connected to other communication links by network switches. These network switches carry such high data bandwidths and switch among so many ports that many separate chassis may be required to contain all parts of the network switch. These many chassis can be separated from each other by tens or even hundreds meters. Optical or electrical signals travelling over cables between chassis can have propagation delays of about 5 nanoseconds (ns) per meter.
A popular network standard for such high-bandwidth optical communication trunks is Synchronous Optical NETwork (SONET). Data is arranged into SONET frames that are timed by a sync pulse that occurs once for each frame, every 125 xcexcsec. For the SONET OC-48 standard operating frequency, one bit is sent every 0.4 ns. A chassis-to-chassis cable delay of 400 ns represents a delay of 1,000 bits. Maintaining the phase of the data to the sync pulse is critical for SONET frames, yet propagation delays are much larger than the data periods. An almost identical format called Synchronous Digital Hierarchy (SDH) is used in most of the world outside of the United States. For purposes of this invention, SONET and SDH are inseparable. Whenever the term SONET is used herein, it is meant to be inclusive of SDH.
FIG. 1 highlights varying data and sync-pulse propagation delays in a multi-chassis network switch. Chassis 12, 14, 15, 16, 18 are elements of network switch 20 that contains several different chassis separated by several meters or even several hundreds of meters. Sync-clock generator 10 generates a sync clock pulse for all chassis elements at network switch 20. The sync pulse is distributed from sync-clock generator 10 to each of chassis 12, 14, 15, 16, 18. The sync pulse is used for timing interface to the external network and to recover the SONET frame at the ingress port and to define the boundaries of the SONET frame at the egress port. It is not used for internal chassis-to-chassis data flow.
A propagation delay occurs from the generation of the sync pulse by sync pulse generator 10 until the sync pulse is received by chassis 12, 14, 15, 16, 18. For example, W chassis 12 is close to sync pulse generator 10 and thus receives the sync pulse after a delay of only 1. However, X chassis 14 is located farther away from sync pulse generator 10 and receives the sync pulse after a delay of 2. Chassis Y is even more remote from sync pulse generator 10 and receives the sync pulse after a longer delay of 4.
Data traveling among chassis 12, 14, 15, 16, 18 also experience propagation delays. For example, data being sent from W chassis 12 to X chassis 14 experiences a delay of 2, as does data traveling in the reverse direction, from X chassis 14 to W chassis 12. Likewise data traveling in either direction between X chassis 14 and Y chassis 16 experiences a delay of 2.
A SONET data stream may originate from two different ingress ports and be combined and output by a single egress port. For example, a secondary stream may be input to chassis 15, while a primary stream is input to chassis 12. The streams from chassis 12, 15 are both sent to chassis 14. The two streams are combined at chassis 14. However, each stream can have a different skew. For example, the primary data stream from chassis 12 has a delay of 3, while the secondary data stream from chassis 15 has a delay of only 2. This relative skew also needs compensation.
FIG. 2A is a table of phase skews between data and the sync pulse in the network switch of FIG. 1. The departure/arrival times of the sync clock and data at chassis W, X, Y of FIG. 1 are shown in the table, for data being sent from chassis W to chassis Y through chassis X. It is assumed that the chassis output the data with little delay.
The sync clock arrives with a delay of 1 at chassis W, a delay of 2 at chassis X, and a delay of 4 at chassis Y. The data is not transmitted from ingress chassis W until the sync clock is received by chassis W, after a delay of 1. The data requires an additional delay of 2 to arrive at chassis X, and another delay of 2 to arrive at chassis Y. Thus the data arrives at chassis X at time 3 (1+2) and at chassis Y at time 5 (1+2+2).
While there is zero skew at ingress chassis W, the skew between the data and sync pulse is 1 at chassis X and Y. In general, when the data travels in the same direction as the sync pulse, the skews are relatively small since the propagation delays for the sync pulse and the data are about the same at any chassis.
FIG. 2B is a table of phase skews between data and the sync pulse in the network switch of FIG. 1 for data traveling in a reverse direction. As before, the sync clock arrives with a delay of 1 at chassis W, a delay of 2 at chassis X, and a delay of 4 at chassis Y.
In this example, data is transmitted in the reverse direction, from chassis Y to chassis W through chassis X. The data is transmitted from chassis Y once the sync clock is received by chassis Y, after a delay of 4. The data requires an additional delay of 2 to arrive at chassis X, and another delay of 2 to arrive at chassis W. The data therefore arrives at chassis X at time 6 (4+2) and at chassis W at time 8 (4+2+2).
The skew at any chassis is the difference in the sync clock arrival time and the data departure/arrival time. For chassis W, the data arrives at time 8, but the clock arrives at time 1 since chassis W is close to the sync clock generator but far from the ingress port (chassis Y). The skew is 7 (8xe2x88x921). For chassis X, data arrives at time 6 while the sync clock pulse arrives at time 2. The difference or skew is 4. Chassis Y, the ingress port, has a zero skew since the data is not transmitted out the egress port until the sync pulse arrives at time 4. Any data arriving at an ingress port of chassis 16 and delivered to an egress port of chassis 16 both use the same delayed sync pulse to recover and send the SONET frame respectively.
Since the data travels in the opposite direction as the sync pulse, the skews are large and increase until the egress port is reached. In a typical network switch, a delay of 1 might correspond to 500 ns, so that the worst-case skew is 3.5 xcexcsec. This is a very large skew.
FIG. 3A shows a SONET frame at an ingress port. Each SONET frame 30 begins with a sync pulse 26. Each SONET frame has an overhead portion 24 and a data payload portion 22. The overhead portion is regenerated at each egress port in each chassis in network switch 20 and is initiated by the sync pulse 26. Data payload portion 22 and overhead portion 24 together contain 810 data bytes divided into 9 rows of 90 bytes in each row. Each frame 30 begins with sync pulse 26, which occurs every 125 xcexcsec.
The data bounded by bytes 34 is known as a Synchronous Payload Envelope (SPE). The SPE 36 floats relative to the boundary of SONET frame 30. Overhead portion 24 includes data pointer 28 that points to the delayed first data byte 34 in Synchronous Payload Envelope (SPE) 36. The delayed first data byte 34 does not have to be at the start of the frame, or the first data byte 32 on the first row of data payload portion 22. Instead, the start of the SPE data can float within a frame. The actual data for the SPE 36 starts with delayed first data byte 34 in the current frame, and extends across the frame boundary into the next SONET frame, and ends before delayed first data byte 34 in the next frame.
Pointers are adjusted to account for timing skew between the network and SONET systems. Pointers can also be adjusted for skew across a SONET system. FIG. 3B shows a SONET frame with pointer adjustment at an egress port. Propagation delays can cause skews between the data and the sync clock. The location of the first data byte 34 can be further delayed relative to the start of the frame 30, creating a relative phase shift of data to overhead. When this occurs, data pointer 28 can be set to the new (delayed) location of delayed SPE first data byte 34xe2x80x2. See U.S. Pat. No. 5,335,223 by Iino, assigned to Fujitsu.
The phase of the SPE 36xe2x80x2 is indicated by data pointer 28 in overhead portion 24 of the output SONET frame. Skews within the switch can cause delayed first data byte 34 of FIG. 3A at the ingress port to be delayed to delayed first data byte 34xe2x80x2 for SONET frame 30 of FIG. 3B when the frame is re-generated at an egress port of the network switch. Data pointer 28 can be adjusted to point to delayed first data byte 34xe2x80x2 at the egress port to accommodate the skew. SPE 36 of FIG. 3A at the ingress port is delayed in phase to SPE 36xe2x80x2 of FIG. 3B at the egress port.
While such pointer adjustment is useful, pointers may need to be adjusted at both the input and egress ports of a large network switch. For a large SONET stream, such as an OC-192, which may contain 192 STS-1 streams, up to 192 pointer adjustments would be needed. This requires a significant amount of logic with its attendant size, cost and power consumption. A large distributed cross-connect may require pointer adjustment at the ingress port to align all cross-connected STS-1 streams prior to cross-connecting. Even so, inter-chassis skew will offset the flows slightly such that they will not all have the same phase relationship with each other and the sync pulse at the egress port, so individual pointer adjustment of each STS-1 stream is again required at the egress to realign the flows.
What is desired is an improved phase adjustment method for a large network switch. It is desired to accommodate various skews when switch elements of a large network switch are located on separate chassis located several hundred meters apart. It is desired to eliminate pointer adjustment at the egress port. A simplified phase adjustment and skew accommodation scheme is desired. Skew tolerance for a SONET network switch is desired. Skew tolerance of data within a SONET frame using a packet-based network switch is desirable.
A skew-tolerant distributed network switch has a plurality of network interface ports that contain ingress ports, egress ports, and switch elements. The ingress ports are coupled to receive input data streams from external sources. The egress ports are coupled to transmit data streams over external links. The switch elements are coupled together and to the ingress and egress ports, forming a switch fabric for switching cell-packets from an ingress port to an egress port.
Each network interface port receives a clock pulse that is synchronized to the data streams. Different network interface ports receive the clock pulse with varying skews to the data streams.
Input buffers in the ingress ports buffer data from the input data streams. A packetizer is coupled to the input buffers. It forms cell-packets from the input data stream. The cell-packets are injected into the switch fabric by the ingress ports and routed to the egress ports through the switch elements.
The input data streams are organized into data frames that comprise one or more of the cell-packets. The clock pulse is for indicating a frame boundary in the input data streams.
Output buffers, in the egress ports, receive and store data from the cell-packets routed through the switch fabric. The cell-packets are re-ordered into a same sequence order as within the input data streams. Transmitters in the egress ports are coupled to the external links. They transmit data from the cell-packets stored in the output buffers over the external links.
Clock triggers receive the clock pulse and are coupled to the transmitters. They delay transmission of the data in the frames until a clock pulse is received by the clock trigger. Thus clock skews are tolerated by delaying transmission of the data from the egress ports until a next clock pulse is received.
In further aspects of the invention the cell-packets from a frame are formed and injected into the switch fabric after a first clock pulse is received by the ingress port, but before the next clock pulse is received by the ingress port. The cell-packets are received by the egress port before the next clock pulse. The next clock pulse is a next pulse after the first clock pulse. Thus the cell-packets are formed and routed through the switch fabric within one clock period.
In other aspects of the invention a header generator is coupled to the packetizer. It attaches a header to each cell-packet. The header includes an internal address of an egress port that the cell-packet is being routed to through the switch fabric, and a sequence number that indicates a sequence order of the cell-packet in the frame from the input data stream. The egress port removes the header from the cell-packet and uses the sequence number to re-order the cell-packet into the same sequence order as the input data stream. Thus internal headers are generated and attached to the cell-packets to indicate the sequence order of the cell-packets.
In still further aspects an offset clock pulse has a same frequency as the clock pulse but also has a phase offset from the clock pulse. Some of the network interface ports receive the offset clock pulse. Cell-packets from a frame are formed and injected into the switch fabric after a first clock pulse is received by the ingress port, but before the offset clock pulse is received by the ingress port. The cell-packets are received by the egress port before the offset clock pulse. Thus the cell-packets are formed and routed through the switch fabric within one clock period adjusted for a skew between the clock pulse and the offset clock pulse.
In further aspects the header of the cell-packets include a frame-sequence number that indicates a sequence order of the frame in a sequence of frames. Thus the frame-sequence number allows for ordering frames when phase offset exceeds one clock period.